For my images, I took a couple shots from my kitchen, standing from the same point but rotating the camera in place.
Afterwards, I used the cpselect tool to define the following correspondence points.
From here, I need to define the Homographic transformation H s.t. $PH=P'$, where p and p' are my two kitchen images. We can solve for H by setting up a system of linear equations in the following form (source):
$$PH = \begin{bmatrix} -x_1 & -y_1 & -1 & 0 & 0 & 0 & x_1x_1' & y_1x_1' & x_1' \\ 0 & 0 & 0 & -x_1 & -y_1 & -1 & x_1y_1' & y_1y_1' & y_1' \\ -x_2 & -y_2 & -1 & 0 & 0 & 0 & x_2x_2' & y_2x_2' & x_2' \\ 0 & 0 & 0 & -x_2 & -y_2 & -1 & x_2y_2' & y_2y_2' & y_2' \\ -x_3 & -y_3 & -1 & 0 & 0 & 0 & x_3x_3' & y_3x_3' & x_3' \\ 0 & 0 & 0 & -x_3 & -y_3 & -1 & x_3y_3' & y_3y_3' & y_3' \\ -x_4 & -y_4 & -1 & 0 & 0 & 0 & x_4x_4' & y_4x_4' & x_4' \\ 0 & 0 & 0 & -x_4 & -y_4 & -1 & x_4y_4' & y_4y_4' & y_4' \\ &&&&...&&&&\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\\ \end{bmatrix} \begin{bmatrix}h1 \\ h2 \\ h3 \\ h4 \\ h5 \\ h6 \\ h7 \\ h8 \\h9 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0\\ 0 \\ 0 \\ 0 \\ 0 \\ 0 \\ ... \\ 1 \end{bmatrix}$$,
where $P$ is an $(2n + 1) \times 9$ matrix and $H$ is an $9x1$ matrix. The solution will give us the $8$ unknowns of $H$, with the last term corresponding to a $1$.
From here, we can input our corresponding points to compute P and then solve the system of equations to find the following transformation:
From our computed transform matrix H, we must use similar techniques as Project 3 to warp the right image to our desired homography (to mosiac with the left image). To do this, we will compute an inverse warp using the inverse of H, and interpreting points from the warping target points to the original image.
Before we rectify our image, we first test our images on a few planar images to straighten them and see if our homographic transformation and warp work as expected. I found these images online (keyboard). Original points in red, projected rectangles in cyan.
As we can see, we are able to rectify the keyboard. We can try to recity another image for demonstration purposes. (source).
Now that we've written warping code and tested it, we can try to blend the original warped kitchen photos. I'll be blending the original left photo with the now warped right photo:
Now we are going to try this entire process on two other images from my .
This part of the project was incredible difficult for me. The underlying mathemathics and linear algebra transformation logic felt fairly straight forward. However, the details of image blending and alignment were incredible confusing, and took tons of research and trial and error to eventually accomplish. I have learned just how much processing goes into the simple panorama's our phones produce, and I've gained an appreciation for the complexity of an effective and efficient solution to what is fundamentally straightforward mathematics.
Note: I have probably spent more hours on Part A of this project than any other project to date...